ELAIS-N1 master catalogue

Checks and diagnostics

In [1]:
from herschelhelp_internal import git_version
print("This notebook was run with herschelhelp_internal version: \n{}".format(git_version()))
This notebook was run with herschelhelp_internal version: 
39887e6 (Tue Jun 27 15:21:21 2017 +0100)
In [2]:
%matplotlib inline
#%config InlineBackend.figure_format = 'svg'

import matplotlib.pyplot as plt
plt.rc('figure', figsize=(10, 6))

import locale
locale.setlocale(locale.LC_ALL, 'en_GB')

import os
import time
import itertools

from astropy.coordinates import SkyCoord
from astropy.table import Table
from astropy import units as u
import numpy as np
from matplotlib_venn import venn3

from herschelhelp_internal.masterlist import nb_compare_mags
In [3]:
OUT_DIR = os.environ.get('OUT_DIR', "./data")
SUFFIX = os.environ.get('SUFFIX', time.strftime("_%Y%m%d"))

master_catalogue = Table.read("{}/master_catalogue_elais-n1{}.fits".format(OUT_DIR, SUFFIX))

I - Summary of wavelength domains

In [4]:
flag_obs = master_catalogue['flag_optnir_obs']
flag_det = master_catalogue['flag_optnir_det']
In [5]:
venn3(
    [
        np.sum(flag_obs == 4),
        np.sum(flag_obs == 2),
        np.sum(flag_obs == 6),
        np.sum(flag_obs == 1),
        np.sum(flag_obs == 5),
        np.sum(flag_obs == 3),
        np.sum(flag_obs == 7)
    ],
    set_labels=('Optical', 'near-IR', 'mid-IR'),
    subset_label_formatter=lambda x: "{}%".format(int(100*x/len(flag_obs)))
)
plt.title("Wavelength domain observations");
In [6]:
venn3(
    [
        np.sum(flag_det[flag_obs == 7] == 4),
        np.sum(flag_det[flag_obs == 7] == 2),
        np.sum(flag_det[flag_obs == 7] == 6),
        np.sum(flag_det[flag_obs == 7] == 1),
        np.sum(flag_det[flag_obs == 7] == 5),
        np.sum(flag_det[flag_obs == 7] == 3),
        np.sum(flag_det[flag_obs == 7] == 7)
    ],
    set_labels=('mid-IR', 'near-IR', 'Optical'),
    subset_label_formatter=lambda x: "{}%".format(int(100*x/np.sum(flag_det != 0)))
)
plt.title("Detection of the {} sources detected\n in any wavelength domains "
          "(among {} sources)".format(
              locale.format('%d', np.sum(flag_det != 0), grouping=True),
              locale.format('%d', len(flag_det), grouping=True)));

II - Comparing magnitudes in similar filters

The master list if composed of several catalogues containing magnitudes in similar filters on different instruments. We are comparing the magnitudes in these corresponding filters.

In [7]:
u_bands = ["WFC u", "CFHT Megacam u"]
g_bands = ["WFC g", "CFHT Megacam g", "SUPRIME g", "GPC1 g"]
r_bands = ["WFC r", "CFHT Megacam r", "SUPRIME r", "GPC1 r"]
i_bands = ["WFC i", "SUPRIME i", "GPC1 i"]
z_bands = ["WFC z", "CFHT Megacam z", "SUPRIME z", "GPC1 z"]
y_bands = ["SUPRIME y", "GPC1 y"]
In [8]:
for band_of_a_kind in [u_bands, g_bands, r_bands, i_bands, z_bands, y_bands]:
    for band1, band2 in itertools.combinations(band_of_a_kind, 2):
        
        basecol1, basecol2 = band1.replace(" ", "_").lower(), band2.replace(" ", "_").lower()
        
        col1, col2 = "m_ap_{}".format(basecol1), "m_ap_{}".format(basecol2)
        nb_compare_mags(master_catalogue[col1], master_catalogue[col2], 
                        labels=("{} (aperture)".format(band1), "{} (aperture)".format(band2)))
        
        col1, col2 = "m_{}".format(basecol1), "m_{}".format(basecol2)
        nb_compare_mags(master_catalogue[col1], master_catalogue[col2], 
                        labels=("{} (total)".format(band1), "{} (total)".format(band2)))
CFHT Megacam u (aperture) - WFC u (aperture) min / max: -8.318923950195312 / 12.368990898132324
CFHT Megacam u (total) - WFC u (total) min / max: -7.089723587036133 / 11.02729606628418
CFHT Megacam g (aperture) - WFC g (aperture) min / max: -5.2399749755859375 / 13.242578506469727
CFHT Megacam g (total) - WFC g (total) min / max: -7.359111785888672 / 13.385894775390625
SUPRIME g (aperture) - WFC g (aperture) min / max: -4.712863922119141 / 47.034759521484375
SUPRIME g (total) - WFC g (total) min / max: -6.363412857055664 / 31.979846954345703
GPC1 g (aperture) - WFC g (aperture) min / max: -5.8217010498046875 / 12.710599899291992
GPC1 g (total) - WFC g (total) min / max: -7.499200820922852 / 7.945899963378906
SUPRIME g (aperture) - CFHT Megacam g (aperture) min / max: -8.120555877685547 / 33.80699157714844
SUPRIME g (total) - CFHT Megacam g (total) min / max: -12.494380950927734 / 34.70111846923828
GPC1 g (aperture) - CFHT Megacam g (aperture) min / max: -13.311277389526367 / 12.847221374511719
GPC1 g (total) - CFHT Megacam g (total) min / max: -10.773859024047852 / 4.7452545166015625
GPC1 g (aperture) - SUPRIME g (aperture) min / max: -47.28156089782715 / 11.266807556152344
GPC1 g (total) - SUPRIME g (total) min / max: -23.504730224609375 / 4.665857315063477
CFHT Megacam r (aperture) - WFC r (aperture) min / max: -5.598031997680664 / 13.370298385620117
CFHT Megacam r (total) - WFC r (total) min / max: -8.268135070800781 / 11.734338760375977
SUPRIME r (aperture) - WFC r (aperture) min / max: -5.611736297607422 / 38.40739822387695
SUPRIME r (total) - WFC r (total) min / max: -8.229608535766602 / 33.31153106689453
GPC1 r (aperture) - WFC r (aperture) min / max: -5.838800430297852 / 9.750600814819336
GPC1 r (total) - WFC r (total) min / max: -7.703699111938477 / 8.159499168395996
SUPRIME r (aperture) - CFHT Megacam r (aperture) min / max: -7.232744216918945 / 33.06208038330078
SUPRIME r (total) - CFHT Megacam r (total) min / max: -8.791389465332031 / 30.27062225341797
GPC1 r (aperture) - CFHT Megacam r (aperture) min / max: -13.84579849243164 / 8.519594192504883
GPC1 r (total) - CFHT Megacam r (total) min / max: -12.175939559936523 / 5.1621246337890625
GPC1 r (aperture) - SUPRIME r (aperture) min / max: -27.661649703979492 / 8.504692077636719
GPC1 r (total) - SUPRIME r (total) min / max: -26.570743560791016 / 2.9275436401367188
SUPRIME i (aperture) - WFC i (aperture) min / max: -7.794872283935547 / 24.732858657836914
SUPRIME i (total) - WFC i (total) min / max: -6.9937286376953125 / 27.258581161499023
GPC1 i (aperture) - WFC i (aperture) min / max: -5.751899719238281 / 8.122200012207031
GPC1 i (total) - WFC i (total) min / max: -9.153600692749023 / 7.193600654602051
GPC1 i (aperture) - SUPRIME i (aperture) min / max: -21.05955696105957 / 7.879997253417969
GPC1 i (total) - SUPRIME i (total) min / max: -10.21229362487793 / 4.299509048461914
CFHT Megacam z (aperture) - WFC z (aperture) min / max: -7.095800399780273 / 11.994922637939453
CFHT Megacam z (total) - WFC z (total) min / max: -7.20233154296875 / 14.031572341918945
SUPRIME z (aperture) - WFC z (aperture) min / max: -6.921276092529297 / 36.78227233886719
SUPRIME z (total) - WFC z (total) min / max: -7.451881408691406 / 38.856048583984375
GPC1 z (aperture) - WFC z (aperture) min / max: -6.590299606323242 / 11.251898765563965
GPC1 z (total) - WFC z (total) min / max: -6.666799545288086 / 11.952099800109863
SUPRIME z (aperture) - CFHT Megacam z (aperture) min / max: -12.261299133300781 / 33.36225891113281
SUPRIME z (total) - CFHT Megacam z (total) min / max: -12.686359405517578 / 26.528608322143555
GPC1 z (aperture) - CFHT Megacam z (aperture) min / max: -12.501282691955566 / 12.042213439941406
GPC1 z (total) - CFHT Megacam z (total) min / max: -14.044147491455078 / 6.308490753173828
GPC1 z (aperture) - SUPRIME z (aperture) min / max: -21.04939842224121 / 11.831649780273438
GPC1 z (total) - SUPRIME z (total) min / max: -9.994834899902344 / 4.559785842895508
GPC1 y (aperture) - SUPRIME y (aperture) min / max: -27.283082962036133 / 10.779624938964844
GPC1 y (total) - SUPRIME y (total) min / max: -26.15351676940918 / 4.937532424926758

III - Comparing magnitudes to reference bands

Cross-match the master list to SDSS and 2MASS to compare its magnitudes to SDSS and 2MASS ones.

In [9]:
master_catalogue_coords = SkyCoord(master_catalogue['ra'], master_catalogue['dec'])

III.a - Comparing u, g, r, i, and z bands to SDSS

The catalogue is cross-matched to SDSS-DR13 withing 0.2 arcsecond.

We compare the u, g, r, i, and z magnitudes to those from SDSS using fiberMag for the aperture magnitude and petroMag for the total magnitude.

In [10]:
sdss = Table.read("../../dmu0/dmu0_SDSS-DR13/data/SDSS-DR13_ELAIS-N1.fits")
sdss_coords = SkyCoord(sdss['ra'] * u.deg, sdss['dec'] * u.deg)

idx, d2d, _ = sdss_coords.match_to_catalog_sky(master_catalogue_coords)
mask = (d2d < 0.2 * u.arcsec)

sdss = sdss[mask]
ml_sdss_idx = idx[mask]
In [11]:
for band_of_a_kind in [u_bands, g_bands, r_bands, i_bands, z_bands]:
    for band in band_of_a_kind:
        
        sdss_mag_ap = sdss["fiberMag_{}".format(band[-1])]
        master_cat_mag_ap = master_catalogue["m_ap_{}".format(band.replace(" ", "_").lower())][ml_sdss_idx]
    
        nb_compare_mags(sdss_mag_ap, master_cat_mag_ap,
                        labels=("SDSS {} (fiberMag)".format(band[-1]), "{} (aperture)".format(band)))
    
        sdss_mag_tot = sdss["petroMag_{}".format(band[-1])]
        master_cat_mag_tot = master_catalogue["m_ap_{}".format(band.replace(" ", "_").lower())][ml_sdss_idx]
        
        nb_compare_mags(sdss_mag_ap, master_cat_mag_ap,
                        labels=("SDSS {} (petroMag)".format(band[-1]), "{} (total)".format(band)))
WFC u (aperture) - SDSS u (fiberMag) min / max: -6.054237365722656 / 8.458396911621094
WFC u (total) - SDSS u (petroMag) min / max: -6.054237365722656 / 8.458396911621094
CFHT Megacam u (aperture) - SDSS u (fiberMag) min / max: -4.2229156494140625 / 14.007747650146484
CFHT Megacam u (total) - SDSS u (petroMag) min / max: -4.2229156494140625 / 14.007747650146484
WFC g (aperture) - SDSS g (fiberMag) min / max: -13.903802871704102 / 3.8203506469726562
WFC g (total) - SDSS g (petroMag) min / max: -13.903802871704102 / 3.8203506469726562
CFHT Megacam g (aperture) - SDSS g (fiberMag) min / max: -4.527894973754883 / 13.139411926269531
CFHT Megacam g (total) - SDSS g (petroMag) min / max: -4.527894973754883 / 13.139411926269531
SUPRIME g (aperture) - SDSS g (fiberMag) min / max: -2.389659881591797 / 27.12592315673828
SUPRIME g (total) - SDSS g (petroMag) min / max: -2.389659881591797 / 27.12592315673828
GPC1 g (aperture) - SDSS g (fiberMag) min / max: -7.5263214111328125 / 10.904970169067383
GPC1 g (total) - SDSS g (petroMag) min / max: -7.5263214111328125 / 10.904970169067383
WFC r (aperture) - SDSS r (fiberMag) min / max: -9.089466094970703 / 5.897480010986328
WFC r (total) - SDSS r (petroMag) min / max: -9.089466094970703 / 5.897480010986328
CFHT Megacam r (aperture) - SDSS r (fiberMag) min / max: -5.955713272094727 / 13.534361839294434
CFHT Megacam r (total) - SDSS r (petroMag) min / max: -5.955713272094727 / 13.534361839294434
SUPRIME r (aperture) - SDSS r (fiberMag) min / max: -3.1653270721435547 / 26.930065155029297
SUPRIME r (total) - SDSS r (petroMag) min / max: -3.1653270721435547 / 26.930065155029297
GPC1 r (aperture) - SDSS r (fiberMag) min / max: -9.684465408325195 / 8.203363418579102
GPC1 r (total) - SDSS r (petroMag) min / max: -9.684465408325195 / 8.203363418579102
WFC i (aperture) - SDSS i (fiberMag) min / max: -7.744133949279785 / 6.387470245361328
WFC i (total) - SDSS i (petroMag) min / max: -7.744133949279785 / 6.387470245361328
SUPRIME i (aperture) - SDSS i (fiberMag) min / max: -3.8502464294433594 / 12.377281188964844
SUPRIME i (total) - SDSS i (petroMag) min / max: -3.8502464294433594 / 12.377281188964844
GPC1 i (aperture) - SDSS i (fiberMag) min / max: -5.126115798950195 / 7.698373794555664
GPC1 i (total) - SDSS i (petroMag) min / max: -5.126115798950195 / 7.698373794555664
WFC z (aperture) - SDSS z (fiberMag) min / max: -4.472288131713867 / 7.737405776977539
WFC z (total) - SDSS z (petroMag) min / max: -4.472288131713867 / 7.737405776977539
CFHT Megacam z (aperture) - SDSS z (fiberMag) min / max: -4.987695693969727 / 12.28380012512207
CFHT Megacam z (total) - SDSS z (petroMag) min / max: -4.987695693969727 / 12.28380012512207
SUPRIME z (aperture) - SDSS z (fiberMag) min / max: -2.913419723510742 / 33.313636779785156
SUPRIME z (total) - SDSS z (petroMag) min / max: -2.913419723510742 / 33.313636779785156
GPC1 z (aperture) - SDSS z (fiberMag) min / max: -6.84623908996582 / 11.156867980957031
GPC1 z (total) - SDSS z (petroMag) min / max: -6.84623908996582 / 11.156867980957031

III.b - Comparing J and K bands to 2MASS

The catalogue is cross-matched to 2MASS-PSC withing 0.2 arcsecond.

We compare the UKIDSS total J and K magnitudes to those from 2MASS.

In [12]:
twomass = Table.read("../../dmu0/dmu0_2MASS-point-sources/data/2MASS-PSC_ELAIS-N1.fits")
twomass_coords = SkyCoord(twomass['raj2000'], twomass['dej2000'])

idx, d2d, _ = twomass_coords.match_to_catalog_sky(master_catalogue_coords)
mask = (d2d < 0.2 * u.arcsec)

twomass = twomass[mask]
ml_twomass_idx = idx[mask]
In [13]:
nb_compare_mags(twomass['jmag'], master_catalogue['m_ukidss_j'][ml_twomass_idx],
                labels=("2MASS J", "UKIDSS J (total)"))
UKIDSS J (total) - 2MASS J min / max: -2.2490740890502927 / 5.001253997802735
In [14]:
nb_compare_mags(twomass['kmag'], master_catalogue['m_ukidss_k'][ml_twomass_idx],
                labels=("2MASS K", "UKIDSS K (total)"))
UKIDSS K (total) - 2MASS K min / max: -2.2481118774414064 / 11.322524047851562
In [15]: